In view of the business significance of stored data, organizations face a challenge to provide data protection and data recovery with the highest level of data integrity. Two primary techniques enabling data recovery are mirroring technology and snapshot technology.
In an extreme scenario of failure (also known as a total crash), the ability to control the transfer of data between the cache memory and the storage space within the storage system, is lost. For instance, all server(s) in the storage control layer could have simultaneously failed due to a spark that hit the electricity system and caused severe damage to the server(s), or due to kernel panic. In this scenario, dirty data which was kept in cache, even if redundantly, will be lost and cannot be recovered. In addition, some metadata could have been lost because metadata corresponding to recent changes was not stored safely, and/or because a journal in which are registered metadata changes between two instances of metadata storing was not stored safely. Therefore, when the server(s) is/are repaired and the storage system is restarted it can be unclear whether or not the stored data can be used. By way of example, because of the lost metadata it can be unclear whether or not the data that is permanently stored in the storage space represents an order-preservation data consistency condition important for crash consistency of databases and different applications.
The problems of crash-tolerant storage systems have been recognized in the contemporary art and various systems have been developed to provide a solution, for example:
U.S. Pat. No. 7,363,633 (Goldick et al) discloses an application programming interface protocol for making requests to registered applications regarding applications' dependency information so that a table of dependency information relating to a target object can be recursively generated. When all of the applications' dependencies are captured at the same time for given volume(s) or object(s), the entire volume's or object's program and data dependency information may be maintained for the given time. With this dependency information, the computer system advantageously knows not only which files and in which order to freeze or flush files in connection with a backup, such as a snapshot, or restore of given volume(s) or object(s), but also knows which volume(s) or object(s) can be excluded from the freezing process. After a request by a service for application dependency information, the computer system can translate or process dependency information, thereby ordering recovery events over a given set of volumes or objects.
U.S. Patent Application Publication Number 2010/0169592 (Atluri et al) discloses methods, software suites, and systems of generating a recovery snapshot and creating a virtual view of the recovery snapshot. In an embodiment, a method includes generating a recovery snapshot at a predetermined interval to retain an ability to position forward and backward when a delayed roll back algorithm is applied and creating a virtual view of the recovery snapshot using an algorithm tied to an original data, a change log data, and a consistency data related to an event. The method may include redirecting an access request to the original data based on a meta-data information provided in the virtual view. The method may further include substantially retaining a timestamp data, a location of a change, and a time offset of the change as compared with the original data.
U.S. Patent Application Publication Number 2005/0060607 (Kano) discloses restoration of data facilitated in the storage system by combining data snapshots made by the storage system itself with data recovered by application programs or operating system programs. This results in snapshots which can incorporate crash recovery features incorporated in application or operating system software in addition to the usual data image provided by the storage subsystem.
U.S. Patent Application Publication Number 2007/0220309 (Andre et al) discloses a continuous data protection system, and associated method, for point-in-time data recovery. The system includes a consistency group of data volumes. A support processor manages a journal of changes to the set of volumes and stores meta-data for the volumes. A storage processor processes write requests by: determining if the write request is for a data volume in the consistency group; notifying the support processor of the write request including providing data volume meta-data; and storing modifications to the data volume in a journal. The support processor receives a data restoration request including identification of the consistency group and a time for data restoration. The support processor uses the data volume meta-data to reconstruct a logical block map of the data volume at the requested time and directs the storage processor to make a copy of the data volume and map changed blocks from the journal into the copy.
U.S. Patent Application Publication Number 2006/0041602 (Lomet et al) discloses logical logging to extend recovery. In one aspect, a dependency cycle between at least two objects is detected. The dependency cycle indicates that the two objects should be flushed simultaneously from a volatile main memory to a non-volatile memory to preserve those objects in the event of a system crash. One of the two objects is written to a stable of to break the dependency cycle. The other of the two objects is flushed to the non-volatile memory. The object that has been written to the stable log is then flushed to the stable log to the non-volatile memory.
U.S. Patent Application Publication Number 2007/0061279 (Christiansen et al) discloses file system metadata regarding states of a file system affected by transactions tracked consistently even in the face of dirty shutdowns which might cause rollbacks in transactions which have already been reflected in the metadata. In order to only request time- and resource-heavy rebuilding of metadata for metadata which may have been affected by rollbacks, reliability information is tracked regarding metadata items. When a metadata item is affected by a transaction which may not complete properly in the case of a problematic shutdown or other event, that metadata item's reliability information indicates that it may not be reliable in case of such a problematic (“dirty” or “abnormal”) event. In addition to flag information indicating unreliability, timestamp information tracking a time of the command which has made a metadata item unreliable is also maintained. This timestamp information can then be used, along with information regarding a period after which the transaction will no longer cause a problem in the case of a problematic event, in order to reset the reliability information to indicate that the metadata item is now reliable even in the face of a problematic event.